Learning of Latent Class Models by Splitting and Merging Components

A problem in learning latent class models (also known as naive Bayes models with a hidden class variable) is that local maximum parameters are often found. This leads not only to suboptimal parameters, but also to a wrong number of classes (components) for a hidden variable. The standard solution of having many random starting points for the EM algorithm is often too expensive computationally. We propose to obtain better starting points for EM by splitting and merging components in models with already estimated parameters. This way we extend our previous work, where only a component splitting was used and the need for a component merging was noticed. We relate a component merging to a model identifiability. We propose an algorithm that learns latent class models by performing a component splitting, merging, and an operation that combines these two. In the experiments with real-world datasets, our algorithm is often better than the standard one when models contain many components. We discuss possible extensions of our work, the most promising of them being to learn cardinalities and parameters of hidden variables in Bayesian networks.